Grouping and Joining Transformations in Data Extraction Process
نویسنده
چکیده
In this paper we present a method of describing ETL processes (Extraction, Transformation and Loading) using graphs. We focus on implementation aspects such as division of a whole process into threads, communication and data exchange between threads, deadlock prevention. Methods of processing of large data sets using insufficient memory resources are also presented upon examples of joining and grouping nodes. Our solution is compared to the efficiency of the OS-level virtual memory in a few tests. Their results are presented and discussed.
منابع مشابه
Grouping and joining transformations in the data extraction process
In this paper we present a method of describing ETL processes (Extraction, Transformation and Loading) using graphs. We focus on implementation aspects such as division of a whole process into threads, communication and data exchange between threads, deadlock prevention. Methods of processing of large data sets using insufficient memory resources are also presented upon examples of joining and ...
متن کاملContourlet-Based Edge Extraction for Image Registration
Image registration is a crucial step in most image processing tasks for which the final result is achieved from a combination of various resources. In general, the majority of registration methods consist of the following four steps: feature extraction, feature matching, transform modeling, and finally image resampling. As the accuracy of a registration process is highly dependent to the fe...
متن کاملIntroducing a method for extracting features from facial images based on applying transformations to features obtained from convolutional neural networks
In pattern recognition, features are denoting some measurable characteristics of an observed phenomenon and feature extraction is the procedure of measuring these characteristics. A set of features can be expressed by a feature vector which is used as the input data of a system. An efficient feature extraction method can improve the performance of a machine learning system such as face recognit...
متن کاملHyperspectral Image Classification Based on the Fusion of the Features Generated by Sparse Representation Methods, Linear and Non-linear Transformations
The ability of recording the high resolution spectral signature of earth surface would be the most important feature of hyperspectral sensors. On the other hand, classification of hyperspectral imagery is known as one of the methods to extracting information from these remote sensing data sources. Despite the high potential of hyperspectral images in the information content point of view, there...
متن کاملAdaptive Approach for Joining and Submissive View of Data in Data Warehouse Using Etl
Data warehouses have emerged as a new business intelligence paradigm where data store and maintain in concurrent. The modifications are required in the implementation of Extract Transform Load (ETL) operations which now need to be executed in an online fashion. The adaptive approach takes two phases. The Extraction phase and the joining phase. The Extraction phase recognition of the subset of s...
متن کامل